DOCUMENT RESUME 



ED 347 774 
TITLiE 

INSTITUTION 
SPONS AGENCY 



PUB DATE 
CONTRACT 
NOTE 

PUB TYPE 

EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



EC 301 392 

Linking Curriculum-Based Assessment *"-o Instructional 
Decision Makings Enhancing Outcomes .or Students at 
Risk for School Failure. Research Progress Series. 
George Peabody Coll. for Teachers, Nashville Tenn. 
John F. Kennedy Center for Research on Education and 
Human Development. 

National Inst, of Child Health and Human Development 
(NIH) , Bethesda, MD. ; Special Education Programs 
(ED/OSERS), Washington, DC. 
May 92 

H023E9G020; HD-15052 
7p. 

Reports - Evaluative/Feasibility (142) 
MF01/PC01 Plus Postage. 

Consultation Programs; Curriculum? -Diagnostic 
Teaching? Elementary Secondary Education; Evaluation 
Methods? Expert Systems; Instructional Development; 
•Measurement Techniques? Student Educational 
Objectives; -Student Evaluation 
-Curriculum Based Assessment 



ABSTRACT 

Curriculum-based measurement (CBM) is described as a 
method of helping teachers measure student performance accurately and 
adapt their instruction to improve student achievement. 
Curriculum-based measurement's features include measurement of 
student proficiency across the annual curriculum, and use of a 
standardized, prescriptive measurement methodology with demonstrated 
psychometric acceptability. Curriculum-based measurement is 
contrasted with the more predominant form of measurement known as 
mastery measurement. The use of CBM to develop effective 
instructional programs and to monitor and adjust goals is described. 
A study which evaluated the us of CBM with and without an expert 
consultation system in math operations is reported, with the finding 
that the instructional changes made by those teachers in the CBM 
group with expert consultation were superior in quality and variety 
to those made by the group without such consul cat ion. Current 
research directions are also described, and a list of nine relate.: 
readings is appended. (JDD) 
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By the year 2000, American students will leave 
grades four, eight, and twelve having demonstrated 
competency in challenging subject matter including 
English, mathematics, science, history, and geography; 
and every school in America will ensure that all students 
leamto use their tmndsweU, so they m^ for 
responsible citizenship, further learning, and productive 
employment in our modern economy. 

— America 2000 National Education Goals 
for All Children 

Lamar Alexander, Secretary 
U.S. Department of Education 

If the nation is to meet its America 2000 goal of 
"demonstrated competency in challenging subject matter," 
teachers must be able to measure student achievement 
accurately and to adjust their instructional methods when 
students are not learning effect^ 
is increasingly important fat two reasons, first* the 
diversity of students in public schools is increasing as the 
rarial and ethnic make-up of the United States is changing. 
Many students have individual learning needs that, if 
ignored* increase their risk for school failure. Second, 
greaser numbers of students with disabilities are receiving 
instruction in general education classrooms. 

The necessary link between instructional decisions 
and student performance traditionally is provided by 
s tanda r di zed achievement tests, but weaknesses assoc i at e d 
with such tests have been well-documented- Some major 
limitations include infrequent measurement, limited match 
between curriculum content and test items, ami lack of 
relevance fat instructional decisions. Instead of using 
standardized tests, many teachers make instructional 
decisions based on unsystematic classroom observations, 
which tend to lack accuracy. 



Kennedy Center investigator Lynn Fuchs* has been 
conducting a long-term research program on Curriculum- 
BasedMeasurement (CB M) that is helping teachers measure 
student performance accurately and adapt their instruction 
to improve student achievement Fuchs's studies on CBM 
have involved general education and special education 
cl?*ses in public schools, and CBM has been implemented 
in mathematics, reading, and spelling- Over the coarse of 
this 15-year research program, the measurement system 
methodology, along with integrated instructional 
recommendation systems, have been refined These systems 
signal teacheis when instructional adjustments are needed 
to improve student performance, and suggest possible 
changes. 



The Curriculum-Based 
Measurement Model 

Curriculum-based measurement has two important 
features: (a) it measures student proficiency across the 
annual curriculum, and (b) it involves a standardized, 
prescriptive measurement methodology, with demonstrated 
psychometric acceptability. 



•Associate Professor of Special Education, Peabody 
College, Vanderbib University. Other Vanderbib facuby involved 
in this research include Douglas Fucks, Professor of Special 
Education and Kennedy Center investigator, and Carol H amies: , 
Research Associate in Special Education; and teachers in 
Nashville Metropolitan Public Schools and the Williamson 
County School System. Thisresearxhhasbeensuppcrtedinpart 
by Grant No. HO23E9OO20 from the Office of Special Education 
Programs, US. Department of Education, 
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To appreciate the distinctiveness of CBM, ii is useful 
to contrast it to the more common, predominant form of 
instructionally related measurement known as mastery 
measurement. In this model, student mastery is described 
on a series of short-term instructional objectives. For 
example, a fourth-grade math teacher's objectives may 
in vol ve bein g able to compute problems, first with multidigi t 
addition, second with multidigit subtraction, third with 
multiplication facts, and so forth. To specify this mastery 
measurement system, a teacher must determine what a 
sensible sequence of instruction, or hierarchy of skills, 
would be, e.g., that instruction will begin with multidigit 
addition with regrouping, followed by multidigit subtraction 
with regrouping. Next, a teacher would design a criterion- 
referenced testing procedure to assess mastery at each step 
in the instructional sequence. The procedure might involve 
25 comparable tests, each containing 10 problems that 
feature multidigit addition with regrouping, with all 
problems involving 3- or 4-digit numerals to maintain a 
comparable level of difficulty . The test would be presented, 
allowing 3 minutes for writing answers, and scoring 
performance in terms of the number of correct problems 
written in 3 minutes. A teacher might define mastery, for 
example, as eight correct problems in 3 minutes on 3 
consecutive days. An instructor would begin by teaching 
multidigit addition with regrouping and then testing, and 
would teach and test until students have mastered this skill. 
Then measurement and instruction would shift to the next 
step in the skill hierarchy, multidigit subtraction. Thus, the 
test changes when the instruction changes. (See Figure 
1A.) 



Assuming the same general goal (proficiency on the 
fourth-grade computation curriculum), a teacher would 
determine the types of problems in the curriculum and their 
relative emphasis. Using randomly generated numerals, 
the teacher would create a series of alternate test forms. 
Each test would comprise 25 problems that represent the 
type and proportion of the problems constituting the fourth- 
grade curriculum. Students would have 3 minutes to 
complete as much of the test as possible; performance 
would be scored in terms of the number of digits correct. 
Each math test samples the year-long domain in the same 
way; each test is an alternate form that represents the 
fourth-grade curriculum. The test samples computation 
performance across the skills representing the curriculum. 

During the first part of the year, a student would likely 
have poor mastery of the curriculum and thus low scores on 
the CBM test, e.g., 27 digits correct. The score does not 
communicate which skills in the curriculum have and have 
not been mastered; rather it indicates that few skills are 
mastered- By February , a student scot? may have increased 
to 51, and by the end of the year to 64. Scores are graphed 
throughout the year, providing a teacher with a visual 
picture of each student's progress. (See Figure 1 B.) 
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A. Example of a mastery 
measurement graph. 



In contrast to mastery measurement, curriculum-based 
measurement has two primary features: (a) it assesses 
proficiency on all skills represented in the year-Jong 
curriculum; (2) it relies on standardized, prescriptive 
measurement methods. 
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Fig. 1 B. Example of a CBM math graph. 

In addition to knowing the rate of student progress, 
teachers need to know which skills students have acquired. 
The CBM database can communicate which skills, or 
problem types, each student in the class has mastered. The 
dolls profile (see Figure 2) provides an easy-to-read picture 
of a student's progress in mastering the range of problem 
types in the year's curriculum- The "Mastery Status" 
section provides data on a student's performance in the 
most recent half month. The "Objectives History" provides 
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Fig. 2. Sample of a CBM Skills Analysis, 

a visual summary of the student* s performance over the 
year, with each block representing a halfmonth. A white 
box indicates that a student did not try this type of problem , 
a striped box thai a student tried the problem type but did 
not do well, a checked box that a student has partially 
mastered the problem type, and a black box that a student 
has successfully mastered the problem type. 



Distinctions Between CBM and 
Mastery Measurement 

1. Scope of skills for measurement. Mastery 
measurement is focused narrowly on a single skill or a 
small cluster of skills at a time. By contrast, CBM is 
focused broadly on a large domain of skills over a year- 
long period CBM may be less sensitive than mastery 
measurement to student change as a result of current 
instruction. However, compared to traditional measurement, 
wh r re performance samples behavior across both grade 
levels and curricula at one moment in time, CBM provides 
information that is sensitive to instructional effects and can 
be used to improve instructional decision making. Because 
CBM ('escribes student performance in terms of proficiency 
on the annual curriculum, both its content and criterion 
validity are stronger than with mastery measurement 

2, Retention and generalization of skills. With 
mastery measurement's close connection between testing 
and instruction, it does not automatically assess retention 



ami generalization of drills. For example, when multidigit 
subtraction is being measured, a teacher does not know 
whether the previously mastered skill of multidigit addition 
has been maintained CBM offers the advantage of 
automatically assessing retention ami generalization of 
skills because each test incorporates all the problem types 
of the year's curriculum. CBM's sensitivity to retention 
and generalization learning may be critical when it is used 
to monitor the development of low-achieving students, 
since they frequently have poorly developed strategies for 
maintaining and transferring skills. 

3. Constancy in measurement across time. Mastery 
measurement requires a shift in measurement each time a 
skill is mastered (see Figure 1 A); CBM maintains a constant 
focus across the year (see Figure IB). With mastery 
measurement, it is impossible to summarize an overall 
learning rate across the different skills in the curriculum, 
because different skills, measured at different times during 
the school year, are not of equal difficulty and do not 
represent equal curriculum units. With CBM, teachers may 
monitor studer f h?*ic skills development across a school 
year without any shifts in measurement Because CBM 
tests sample across the entire year-long curriculum, test 
difficulty remains constant across the school year. The 
constancy associated with CBM permits summaries of 
student learning rates across time. The CBM database can 
be used to compare the effectiveness of different 
instructional components introduced at different times 
during the year. 

4. Reliance on instructional hierarchies. The 
structure of mastery measurement specifies the order in 
which instruction must proceed, and one cannotprogress to 
subsequent skills until mastery of the current skill is 
demonstrated. Moreover, the mastery measurement 
framework typically results in a skills-oriented approach to 
instruction. With mastery measurement, instruction (the 
independent variable) and measurement (the dependent 
variable) are tied together, with both simultaneously focused 
on skills. With CBM, the current instructional focus or 
procedure (the independent variable) is not tied to ami 
determined by measurement ( he dependent variable); 
therefore, measurement and instruction are not confounded. 
Because of this, CBM offers the advantage of permitting 
tcichers to experiment with contrasting instructional 
chunks, sequences, and procedures. Teachers use the CBM 
database as the dependent variable by which they evaluate 
the effectiveness of contrasting instructional strategies. 

5. Development of tests. Mastery measurement relies 
primarily on the use of teacher-made criterion-referenced 
tests, whose psychometric characteristics are unknown. 
Even when teachers rely on commercial criterion-referenced 
tests for mastery measurement psychometric characteristics 
are uncertain. By contrast a standard CBM methodology 
has been formulated. When teachers have determined the 
curriculum they expect students to master over the course 
of the school year, CBM prescribes methods for creating, 
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administering, scoring, and using tests that result in reliable 
ami valid descriptions of students* growth in reading, 
spelling, written , xpression, and math operations and 
applications 

Using CBM to Develop Effective 
Instructional Programs 

Fuchs's research shows that CBM does index student 
progress and achievement accurately and that, when used 
in certain ways, it can help teachers plan better instructional 
programs (Fuchs, Deno, & Mirkin, 1984). 

Research supports three strategies for using CBM to 
assist teachers in developing instructional programs. First, 
teachers can use CBM to monitor the appropriateness of 
the goals they set and to ensure the use of realistic, but 
ambitious, goals(Fuchs,Fuchs f & Ham leu, 1989a). Second, 
CBM can be used to determine the adequacy of student 
progress, to determine whether instructional programs 
require adjustment, and to compare the effectiveness of 
alternative programmatic components (Fuchs, Fuchs, & 
Hamlett, 1989b). Finally, CBM databases can be used to 
draw profiles of strengths and weaknesses, in order to 
assist teachers in determining the nature of effective 
programmatic modifications (Fuchs, Fuchs, Hamlett, & 
Sleeker, 1990). (For detail on these three strategies, see 
Fuchs, in press b.) 

Using CBM to Monitor 
and Adjust Goals 

Research substantiates the effectiveness of using 
goals to improve instructional outcomes. Within typical 
CBM practice, when a student's actual rate of progress 
falls below the rate necessary for goal attainment, the rate 
of a student's progress and the effectiveness of the 
instructional program are judged inadequate, and CBM 
decision rules recommend a teaching change- Thus, the 
performance criterion specified in the goal becomes critical 
in the instructional decision-making process. When teachers 
set goals that are unambi tiously low, as is often the case for 
students with disabilities or other low-achieving students, 
few if any recommendations for instructional improvements 
will be made. Fuchs's research with CBM indicates that the 
level of goal ambitiousness, not goal attainment, is 
associated with student achievement. 

Fuchs has explored dynamic goal setting as one 
potential solution to the problem of unambitious goals. 
Each of 30 special education teachers who taught self- 
contained and resource programs for students in grades 2 
through 9 selected two mildly handicapped students with 
math goals in their Individualized Education Plans. Teachers 
were assigned randomly to three treatment groups: dynamic 
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goal CBM, static goal CBM, and control. The control 
teachers monitored student progress using conventional 
special education practice, including unit tests, correction 
of assignments, and observation erf student performance. 
The teachers in both CBM groups, over 15 weeks, used 
CBM to track their two pupils ' progress toward math goals. 
Within the static goal CBM group, when a student's actual 
rate of improvement exceeded the rate anticipated in the 
goal line, the decision read 4 'OK! Collect more data*' The 
data pattern suggested that the student's rate of progress 
was acceptable with respect to goal attainment, and that the 
corresponding instructional program looked effective. The 
teachers always were free to increase the goal, but they 
were never directed to do so. By contrast, within the 
dynamic goal CBM group, when a student's actual rate of 
improvement exceeded the rate anticipated in the goal line, 
the decision read "OK! Raise the goal to X** (where X = the 
student's predicted performance at the aid of the study, 
based on the student's current rate of progress). 

With respect to use of goals, teachers in the dynamic 
goal CBM group made more goal increases than did 
teachers in the static goal CBM group. Most dramatic was 
the size of the effect Within the dynamic goal group, 
teachers increased goals for more than (Hie out erf every two 
pupils; in the static goal group, only one teacher, for one of 
her pupils, spontaneously increased a goal in response to 
the student's data. This finding suggests that despite the 
potential importance of ambitious goals, special educators' 
typical goal-setting standards may underestimate many 
students' potential . It also suggests that without systematic 
prompting to raise goals, practitioners cannot be expected 
to do so. 

A second major finding was that differential student 
achievement was concurrent with teachers' goal-raising 
behavior. Students in the dynamic goal CBM group achieved 
better than the controls during posttesting on a standardized 
math operations achievement test. However, the 
achievement of the static goal CBM group did not exceed 
Lhat of the controls. Given the finding that such goal 
adjustment and goal ambitiousness may enhance student 
achievement, the special education community might 
consider adoption of CBM systems that incorporate dynamic 
goal-setting procedures. 

CBM With and Without an Expert 
System in Math 

Fuchs's research had demonstrated that CBM with 
consultation works better than consultation alone (Fuchs et 
aL, 1984), but the effect of CBM when consultation was not 
in place was not known, The research team also was 
interested in finding a wr>y to deliver consultation in a 
somewhat standardized way, and in a way that would be 
less expensive than frequent consultant visits. An "expert 
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system" was designed in math operations to provide thai 
consultation. (An expert system is a computer program that 
tries to reproduce the advice that an export might provide.) 
The research study addressed the question: What are the 
effects of an objective ongoing assessment system with 
and without "expert" instructional consultation on the 
amount and type of teacher instructional adaptation and on 
student achievement? 

The expert system considered: information from the 
CBM database on individual and collective student 
performance; information about previous instructional 
programs (what has been tried with students) ami the 
effects those programs had on student performance; teacher 
judgments about student work habits and motivation; and 
teacher cumcular priorities. Eased on this information, the 
expert system recommended what skills to work on next 
am! alternative strategies for working on those skills. When 
appropriate, the expert system also recommended strategies 
far maintaining skills and for enhancing student motivation. 

The study involved 33 special education teachers 
who had been randomly assigned to 3 groups; there were 
1 1 teachers and close to 22 students in eachgroup. The three 
groups were: CBM with expert consultation; CBM without 
consultation; and a control group without CBM. The 
intervention lasted for 20 weeks. 

The following example serves to illustrate the process. 
Initially f a student was scoring 35 digits correct (Hi the math 
operations curriculum, and the teacher set a goal of 57 
digits correct Software was used to graph the approximate 
rate of progress that the student needed to demonstrate in 
* order to meet the goal in the time frame the teacher had set. 
If student progress was good, a recommendation directed 
the teacher to raise the goal, and a new progress rate was 
projected Afteramonth and a half, however, the assessment 
indicated that the student was not achieving the goal; so an 
instructional adjustment was recommended. A teacher in 
the group with expat system consultation would interact 
with the system to gel suggestions about how to modify 
instruction. A teacher in the group without expert system 
consultation would develop alternative ideas independently. 
Teachers in both groups had access to studen t skills analyses. 

Teachers in the CBM groups with and without 
consultation adapted instruction more frequently than did 
teachers in the control group, Teachers in the CBM groups 
made between two and three major instructional changes 
over the course of the intervention , but student achievement 
was better in the CBM group with consultation (an effect 
size of approximately one standard deviation, which is a 
reliable and practically important difference). (For more 
detail, see Fuchs f Fuchs, Hamlett, & Stecker, 1991.) 

An important finding was that the instructional 
changes were superior in quality and variety in the CBM 
group with expat consultation. Teachers in this group 
taught a greater variety of problem types, especially 
multidigit subtraction, often a problem area for low- 
achieving students. They encouraged students to use "self- 
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talk" strategies in solving problems. Teachers used a greater 
variety of algorithms, or different strategies, for teaching a 
given problem type. They used mixed problem types more 
frequently to maintain skills. Teachers in the nonexpert 
system groups relied on relatively low forms of instructional 
adaptation, essentially re-explaining or reviewing how to 
solve a problem, or using the same algorithm or strategy 
used previously but giving more practice time. 

Findings in studies involving CBM with expert 
systems in math, reading, and spelling have been content 
specific, but across the three domains this finding was 
consistent: Without expert systems, teachers focus 
instructional changes on what to teach or re-teach. With 
expert systems, teachers focus on what and how to teach, 
i.e., not only content but also strategy or process. 

Current Research Directions 

Because of teachers' logistical problems in 
implementing many programmatic changes for different 
students at different times, Fuchs and her colleagues are 
developing and evaluating techniques for simultaneously 
considering the assessment information of all students in a 
class. These techniques focus on presenting classwide 
graphs and skills profiles and making instructional 
suggestions for groupings of students. The goal is to 
provide recommendations to teachers, which will improve 
whole-class instruction while at the same time tailoring 
some aspects of instruction for individual student needs, in 
both general education and special education classrooms. 

Recommendations include the use of classwide pea 
tutoring, which provides a classroom organizational 
structure by which students in need of remediation in 
certain skill areas can be assisted by those who have 
mastered the specific skills. Over a period of time, the 
system can ensure that all students have opportunities to be 
"coaches" (i.e., tutors) as well as "players" (i-e,« tutees). 
Preliminary findings suggest the potential power of the 
classwide applications for enhancing achievement outcomes 
for students with mild and moderate academic problems 
(Fuchs t in press a). Additionally, peer tutoring has proved 
to be a workable ami effective method for providing 
differentiated instruction, where students receive formal 
instruction on different skills, within a large-group 
instructional setting. 

The longstanding goal of Fuchs* s research program 
on CBM is to investigate methods for integrating the use of 
ongoing assessment information with instructional 
planning, so that teachers can begin to address the diverse 
needs of their students effectively. The hope is that providing 
teachers with accurate assessment information, along with 
feasible routines for using that information to differentiate 
instruction , will increase our country *s capacity to meet the 
challenge of helping American students ! ^ave school with 
academic competence, prepared for responsiblecitizenship 
ami productive employment 
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